Video based Activity Recognition in a Trauma Center

We present a feasibility study of automated, vision-based detection and recognition of trauma procedures in a medical emergency room. Given a ceiling-mounted camera view of the trauma room, our goal is to track and transcribe the activities performed during resuscitation of a patient, the time instances of their initiation and their temporal durations. We represent activities through complex spatio-temporal relationships between image features based on scene dynamics, patient localization, clinicians’ hand motions and medical devices. We design an activity grammar based on trauma specific domain-knowledge and model the resulting logic as a Markov Logic Network. Probabilistic inference of activity posterior is computed efficiently in the presence of observed features. To this end, we demonstrate our approach on videos of realistic trauma simulations in challenging, multi-agent. multi-task settings. This study primarily aims at exploring the overall problem of visual recognition of trauma procedures. The accuracy of the results we obtained from our recognition scheme confirms the suitability of our framework.